Upgrade CI #382
Upgrade CI #382
Conversation
gwynne
commented
Aug 5, 2023
gwynne
commented
- Improves naming conventions a little
- Factors out code coverage into its own single job (results in overall speed improvement for CI)
- Adds workflow for analyzing the code with GitHub's CodeQL. Initial commit purposely leaves out the additional permission requirement for the token so it can't open any security complaints on the repo yet.
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #382 +/- ##
=======================================
Coverage 46.88% 46.88%
=======================================
Files 110 110
Lines 8988 8988
=======================================
Hits 4214 4214
Misses 4774 4774 |
|
This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation. |
.github/workflows/test.yml
Outdated
| code-coverage: | ||
| runs-on: ubuntu-latest | ||
| container: swift:jammy | ||
| steps: | ||
| - name: Check out code | ||
| uses: actions/checkout@v3 | ||
| - name: Run unit tests for coverage data | ||
| run: swift test --filter=^PostgresNIOTests --enable-code-coverage | ||
| - name: Upload coverage data | ||
| uses: vapor/[email protected] | ||
|
|
There was a problem hiding this comment.
why did you move code cov into its own job? why isn't this part of unit testing anymore?
There was a problem hiding this comment.
Because I did some digging into what actually results from providing the code coverage from every unit test variant. The results are always identical (except where there are Swift- or DB-version dependent semantics that are touched by unit tests, which is extremely rare right across all of Vapor). In short, it slows down each individual variant (both the build and the execution, enough to be easily measurable in some cases) and provides nowhere near enough valuable data to be worth the trouble even in CI. Splitting it out to its own run adds another job, yes, but that job doesn't need to have TSan enabled; in most repos the end result is the overall time taken to run CI goes down, despite using the additional queue slot.
And on a minor note, Codecov's backend seems to appreciate not having to process 20 identical uploads for every single push to a PR; the upload failure rate drops from ~4% to literally zero (at least as observed thus far). On, and for good measure, the additional env vars that were being included with the uploads don't seem to be visible anywhere in Codecov's interface, so even where there were differences between coverage on a per-variant basis, you couldn't tell which was which anyway.
